Diving deeper into mentee networks
نویسندگان
چکیده
Modern computer vision is all about the possession of powerful image representations. Deeper and deeper convolutional neural networks have been built using larger and larger datasets and are made publicly available. A large swath of computer vision scientists use these pre-trained networks with varying degrees of successes in various tasks. Even though there is tremendous success in copying these networks, the representational space is not learnt from the target dataset in a traditional manner. One of the reasons for opting to use a pre-trained network over a network learnt from scratch is that small datasets provide less supervision and require meticulous regularization, smaller and careful tweaking of learning rates to even achieve stable learning without weight explosion. It is often the case that large deep networks are not portable, which necessitates the ability to learn mid-sized networks from scratch. In this article, we dive deeper into training these midsized networks on small datasets from scratch by drawing additional supervision from a large pre-trained network. Such learning also provides better generalization accuracies than networks trained with common regularization techniques such as l2, l1 and dropouts. We show that features learnt thus, are more general than those learnt independently. We studied various characteristics of such networks and found some interesting behaviors.
منابع مشابه
Maximum Dive Depths of Eight New Zealand Procellariiform including Pterodroma Species
Lightweight capillary tube depth gauges were attached to breeding at New Zealand colonies during the period 1998-2008. This paper presents the first information on the diving ability of Pterodroma Grey-faced Petrels, Pterodroma macroptera gouldi, recorded maximum dives down to 23 m. Males (6.3 ± 6.3 m SD) dived deeper on average than females (3.6 2.5 m) during the incubation period but not sign...
متن کاملDiel Oscillations in Whale Shark Vertical Movements Associated with Meso- and Bathypelagic Diving
—A female whale shark Rhincodon typus satellite tagged off the coast of Mozambique and moving to the east coast of Madagascar where the tag popped off after 87 d showed increased mesoand bathypelagic diving while crossing the bathymetrically nonconstraining southern part of the Mozambique Channel. After three weeks with daily maximum diving depths in the epipelagic zone at the beginning of the ...
متن کاملFlipper stroke rate and venous oxygen levels in free-ranging California sea lions.
The depletion rate of the blood oxygen store, development of hypoxemia and dive capacity are dependent on the distribution and rate of blood oxygen delivery to tissues while diving. Although blood oxygen extraction by working muscle would increase the blood oxygen depletion rate in a swimming animal, there is little information on the relationship between muscle workload and blood oxygen deplet...
متن کاملNetworks in cognitive systems and biomedicine: cerebral processes, models and mathematical tools design.
Convergence of clinical, empirical, methodological and theoretical approaches aimed at understanding the relation between brain function and cognition, is by now standard in most if not all academic programs in the area of Cognitive Science. This confederation of disciplines is one of the liveliest domains of inquiry and discussion into some of the most fundamental--and historically resilient--...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1604.08220 شماره
صفحات -
تاریخ انتشار 2016